PAutomaC: a PFA/HMM Learning Competition

Approximating distributions over strings is a hard learning problem. Typical techniques involve using finite state machines as models and attempting to learn these; these machines can either be hand built and then have their weights estimated, or built by grammatical inference techniques: the structure and the weights are then learned simultaneously. The PAutomaC competition, run in 2012, was the first challenge to allow comparison between methods and algorithms and built a first state of the art for these techniques. Both artificial data and real data were proposed and contestants were to try to estimate the probabilities of new strings. The purpose of this paper is to describe some of the technical and intrinsic challenges such a competition has to face, to give a broad state of the art concerning both challenges dealing with learning grammars and finite state machines and the relevant literature. This paper also provides the results of the competition and a brief description and analysis of the different approaches the main participants used. S. Verwer Institute for Computing and Information Sciences, Radboud University Nijmegen. E-mail: [email protected], R. Eyraud QARMA team, Laboratoire d’Informatique Fondamentale de Marseille. E-mail: [email protected] C. de la Higuera TALN team, Laboratoire d’Informatique de Nantes Atlantique, Nantes University. E-mail: [email protected] 2 Sicco Verwer et al.